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Abstract 

A  survey  of  some  evaluation  work  on  discrete  reliability  growth  models 
is  presented.  Extension  of  an  accurate  exponential  growth  model  is  pro- 
vided that  uses  regression  analysis  to  fit  the  natural  logarithm  of  the  failure 
rate  1  —  p  in  the  geometric  distribution.  Some  useful  theorems  and  relation- 
ships are  developed  that  provide  estimates  of  reliability  which  have  better 
properties  than  the  usual  maximum  likehood  estimates.  The  effect  of  dis- 
counting is  portrayed  with  graphs  that  allow  comparison  among  different 
failure  discounting  methods  and  their  affect  on  different  models. 

Keywords:  Reliability  Growth,  Failure  Discounting,  Reliability  Esti- 
mation. 

1      SUMMARY 

Reliability  growth  models  and  their  use  in  the  acquistion  of  systems  by  DOD 
agencies  have  become  more  important  in  recent  years.  They  are  used  not  only 
to  track  reliability  growth,  but  also  to  verify  reliability  requirements.  It  is  im- 
portant that  these  models  track  reliability  change  accurately.     This  tracking 


accuracy  can  be  determined  by  application  of  computer  simulation.  The  re- 
sults of  simulating  several  reliability  models  that  utilize  discrete  test  data  are 
reviewed  in  this  paper. 

How  to  dispose  of  past  failures  after  their  cause  has  been  removed  by  appro- 
priate changes  in  design  or  production  has  been  a  problem  of  interest  for  many 
years.  Removal  of  a  failure,  or  a  fraction  of  a  failure,  from  the  test  data  is  called 
failure  discounting. 

The  results  show  that  discounting  definitely  affects  the  accuracy  of  a  relia- 
bility growth  model.  Certain  types  of  failure  discounting  improve  the  accuracy 
of  some  reliability  growth  models  analyzed.  One  type  of  failure  discounting, 
which  is  in  use  today,  decreases  the  accuracy  of  some  reliability  growth  models 
analyzed. 

2  BACKGROUND  AND  FOUNDATIONS 

Let  X(r)  denote  number  of  independent  trials  to  r  failures  with  probability  p 
of  success  on  each  trial.  The  probability  distribution  of  X(r)  is  the  well  known 
Pascal  distribution  with  probability  mass  function: 

/(x)=(r-l)pX_r(1-p)r         *  =  r,r  +  lf.... 

We  say  X(r)  is  P(p,r)  to  denote  this  probability.  For  r  =  1,  X(l)  denotes 
trials  to  first  failure.  If  Xj(l)  denotes  trials  between  failure  numbers  j  —  \  and 
j,  j  =  1,2,..., r,  then, 

X(r)  =  X1(l)  +  X2(l)  +  ..-  +  Xr(l). 

Mission  testing  is  frequently  performed  on  items  until  r  failures  occur,  at 
which  time  some  change  is  made  that  hopefully  will  increase  the  probability  p  of 
success  for  subsequent  trials.  Let  pk  denote  the  probability  of  success  (reliability) 
after  modification  number  k.  Thus,  po5Pi>  ■  ■  •  ■>  denote  reliabilities  of  the  items 


UUUUtlT  rv>»w/\  i_n-»i>/- 


under  test  in  phases  0, 1,2, . .  ..  We  can  model  pk  as  follows: 

pk     =i_c-(a+6*)j  a  +  6A:>0,    k  =  0,1,2,... 

(1) 
=  1  -  e~Ak. 

This  model  has  become  known  as  the  discrete  exponential  reliability  growth 
model.  It  first  appeared  in  reference  (1).  It  was  first  evaluated  by  computer 
simulations  in  reference  (3).  In  each  phase,  we  seek  current  estimates  for  a  +  bk 
using  test  data  from  the  previous  and  current  phases.  In  this  model  a  +  bk  = 
—  In  (1  —  pk)-  Since  1  —  pk  is  the  failure  rate,  the  problem  is  one  of  fitting  the 
negative  natural  logarithm  of  the  failure  rate.  Regression  methods  have  been 
used  to  make  this  fit.  One  such  method  is  described  in  detail  in  Section  4. 

Some  readers  may  prefer  to  estimated  pk  directly  using  well  known  estima- 
tion results  for  this  class  of  probability  distributions.  Since  X(r)  is  a  complete 
sufficient  statistic  for  pk  ,  it  can  be  shown  that  a  uniformly  minimum  variance 
unbiased  estimator  for  pk  is 


Pk  -  \ 


0  if  X(r)  =  r 


In  development  programs  it  may  be  desireable  to  take  r  =  1.   In  this  case  the 
uniform  ly  minimum  variance  unbiased  estimator  for  pk  is 

(   1     ifX(l)  >  1 

Pk  =  { 

{  0    ifX(l)  =  0 

This  is  hardly  a  desirable  estimator  since  pk  will  never  be  0  or  1. 

In  any  event  some  model  is  needed  for  pk  that  reflects  changes  in  hardware 
reliability  due  to  changes  in  the  acquisition  process.  Since  changes  can  improve 
or  degrade  reliability,  a  model  (including  the  estimation  procedures)  should  have 
the  ability  to  track  both  degrading  reliability  and  reliabililty  growth. 


2.1      Point  Estimation  Using  Trials  Between  Failures 

Suppose  X  is  P(p,  1)  and  p  =  1  —  e~A.  Then  the  probability  mass  function  f(x) 

{  p'-^l-p),    i  =  l,2,... 

0,  elsewhere. 

The  maximum  likehood  estimator  (MLE),  p*,  for  p  is  p*  =  (X  —  1)/X.  This 
estimator  has  negative  bias.  An  estimator  with  less  negative  bias  is  p  —  1  —  e~A, 
where  A  is  unbiased  for  A.  A  is  obtained  as  follows:  From  equation  (1), 

2  3 

A=-ln(l-p)=p+|-  +  |-  +  .... 

If  5(1)  is  unbiased  for  A  then, 

00  n2        n3 

Y,g(x)p*-\l-P)  =  p+^  +  ^  +  ....  (2) 

z=l 

The  left  member  of  equation  (2)  yields 

(1-p)  [g(l)  +  g(2)p  +  g(Z)p2  +  ...]=  g(l)+p(g(2)-g(l))+P2(g(Z)-g(2))  +  . . . . 

Equating  coefficients  of  this  polynomial  with  those  of  the  right  member  of  equa- 
tion (2)  we  obtain 

[0,  X  =  l 

9(X)=\  1  1  (3) 

{1+2  +  --+X^V    X>1 

Therefore,  A  =  g(X)  and  p  =  1  —  e~A.   Of  course  p  is  not  unbiased  for  p,  but 
the  following  theorem  is  useful. 


Theorem  1:  If  X  is  P{p,l),  A  =  g(X),  p  =  1  -  e~A,  p*  the  MLE  for  p  then 

1  

1.  E{p*)  =  1+ -ln(l  -p) 

P 

2.  p*  <  p  for  all  X  >  1 


3.  E(p*)  <  E(p)  <  p 

where  g(X)  is  defined  by  equation  (3).  The  proof  follows: 
1.   E(p*)  <  p: 

X-l  1 

v  X  X 


E 


X 


oo     1 


Thus, 


=  (i-p)  +  |p(i-p)  +  ip!(i-p) 


F      2        2         3         3 


1 


ln(l-p)--ln(l-p). 

P 


E{p*)     =E(l-±)=l-\n(l-p)  +  -\n{l-p) 


2  2 

P        P  P 

=  p-  -+  —  -—...  <p 

^223  F 


2.  p*  <  p  for  all  x  >  1:  (by  induction) 


(4) 


Since  ex  >  1  -| —  for  all  x  >  1,  then 
x 


1  ,  /  1 
-  >  In  1  +  - 
x  \        x 


(5) 


a)  for  x  =  2,  p*  =  -,  p  =  1  -  e_1  =  .632 


b)  suppose  for  some  interger  x  >  2,  p*  <  p.  Then, 


1 <  1  —  exp 

x 


1  1 

!+  «  +  ••■+ 7 

2  x-l 


(6) 


i.e. 

In    *<l  +  I  +  ...  +  -i_  (7) 

From  equation  (5) 

In   (l  +  -  J  =  In  (x  +  1)  -  In  (x)  <  -  (8) 

adding  equation  (7)  and  (8)  we  obtain 

ln(z  +  l)<  1  +  -  +  ...  +  -.  (9) 

2  x 

This  establishes  (6)  and  (7)  for  all  integers  >  1.  Therefore,  p*  <  p  for  all  x  >  1. 
3.  E{p*)  <  E(p)  <  p: 

p*  <  p     implies      E(p*)  <  E(p)  . 

Also  p{A)  =  1  -  e~A  is  convex  up  in  A  and  p{A)  =  p.  The  derivative  of  p(A) 
at  A  =  A  is  e  and  is  the  slope  of  the  tangent  line  to  p(A)  at  A  =  A.  This 
tangent  line  is 

L(A)  =  p(A)  +  e"A(A  -  A)  =  p  +  e~A{A  -  A). 

From  convexity  of  p(A), 

p(A)  <  L(A). 

with  equality  only  at  A  =  A.  Therefore 

E(p(A))  <  E[L(A)}  =  L(E(A))  =  p. 

Suppose  X(r)  is  P(p,r,)  and  X{,  i  =  l,2,...,r  denotes  the  trials  between 
failures.  Then  A,  =  1  +  |  +  . . .  +  ^■1_1  is  unbiased  for  A  =  —  In  (1  —  p).  Conse- 
quently, A  =  =  Y7\  A-i  is  also  unbiased  for  A.  The  estimator  p  =  1  —  exp(  — A)  is 
an  improved  estimator  for  p.  We  shall  see  in  section  2.4  that  there  is  a  better 
estimator  for  A  using 

J2Xi  =  X(r). 


2.2      Characteristrics  of  p  and  p* 

Even  though  E(p*)  <  E(p)  <  p,  p  may  not  be  preferable  to  p*,  unless  var(p)  = 
var(p*).  More  specifically,  it  would  be  desireable  to  have  the  mean  squared 
errors  compare  favorably:  i.e.,  we  would  like 

E(p-p)2<   E(p*-p)2 

for  values  of  p  in  the  ranges  of  interest.  Expressions  for  the  mean  and  the 
variance  of  p*  are  as  follows: 

1- 


E(fi*) 


1  + 


-)ln(l-p) 


1\  ,„  ,  1-P^   1 

=  var[p*)  2_^  ~2 


1 


ln(l-p) 


p    T x'         v    p 

The  mean  and  variance  of  p  will  be  approximated.  It  is  of  interest  to  note  that 

oo      n 


var. 


(see  ref(l)).  Table  1.  presents  some  values  of  p,  E(p*),  E(p),  var(p*),var(p), 
MS E(p*),  MSE(p).  Most  of  the  values  were  obtained  from  computer  simula- 
tions using  10,000  observations.  Table  2.  affords  a  comparison  of  p*  with  p  for 
various  values  of  X. 


p 

E(p*) 

E(P) 

var(p*) 

var(p) 

MSE(p*) 

MSE(p) 

.5 

.307 

.357 

.102 

.137 

.141 

.157 

.6 

.389 

.451 

.112 

.143 

.157 

.166 

.7 

.484 

.547 

.115 

.140 

.163 

.163 

.8 

.598 

.661 

.107 

.121 

.149 

.140 

.9 

.744 

.797 

.079 

.080 

.103 

.090 

.95 

.842 

.881 

.051 

.047 

.062 

.052 

.97 

..892 

.921 

.035 

.031 

.040 

.033 

.99 

.953 

.968 

.014 

.012 

.015 

.012 

Table  1:  Statistical  Characteristics  of  p*  and  p 


X(l) 

p* 

P 

5 

.800 

.875 

10 

.900 

.941 

20 

.950 

.971 

150 

.993 

.996 

Table  2:  Values  of  p*,  p  and  X(l) 

2.3      Point  Estimation  Using  X(r) 

When  testing  items  until  r  fail,  the  number  of  trials  between  failures  X\,  ..., 
Xr,may  or  may  not  be  recorded.  Either  way  X(r)  is  observed  and  it  is  a  complete 
and  sufficient  statistic  for  p.  It  is  also  sufficient  for  —  ln(l  —  p);  i.e.  for  A. 
Therefore,  there  is  an  unbiased  minimum  variance  estimator  g(X(r))  for  A.  The 
unbiased  estimator  A  discussed  after  Theorem  1,  is  unbiased,  but  may  have 
larger  variance  or  mean  squared  error  than  g(X(r)).  Since  X(r)  is  P(p,r),  we 
could  construct  g(X(r))  as  before,  by  writing 

'i-l\ 


£*(*) 


r-  1 


•(i-p)'=p+£  +  £  +  . 


and  equate  coeffecients  of  both  members  of  this  equation.  Instead,  we  construct 
g(X(r))  using  recursive  relationships. 


Theorem  2:  If  X(r)  is  P(p,r),  p=  1  -  e~A ,  then 

0 


1  1 

-  +  — TTT  +  ..-  + 


1 


if  X{r)  =  r 
if  X(r)  >  r 


r       r+1  X(r)  -  1 

is  a  minimum  variance  unbiased  estimator  for  —  In  (1  —  p)  =  A. 


Proof: 

Suppose  X(r)  is  P(p,r)  and,  let  gr(X(r))  be  an  unbiased  estimator  for 
-ln(l  -  p).  Likewise,  for  X{r  +  1),  let  hr(X{r  +  1))  =  gr+i{X(r  +  !))■  We 
shall  drop  subscripts  and  write 

X  is  P(p,r),  Y  is  P(p, r  +  1) 


and  let  g{X)  and  h(Y)  be  unibiased  estimators  for  —  ln(l  -  p). 
Case  1:  r  =  1 

We  know  from  equation  (3)  that 

0  if  X  =  1 


!(*)=  1 


1  +  o  +  ■  •  •  + 7     if  X  >  1 

2  x  -  1 

Since  both  g(X)  and  h(Y)  are  unbiased  for  the  same  quantity, 
oo  oo  /  1    \ 

(i-p)E^)px_1  =  (i-p)2EM*)     x7       pz~2-  (io) 

Z=l  2=2  V  / 

Dividing  out  (1  -  p),  expanding  the  resulting  R.H.S.  of  (9),  transposing  one  of 
the  resulting  series  to  the  L.H.S.,  and  using  the  fact  that  g(l)  —  0,  we  have 

oo  oo 

£  [g(x)  +  (x-  l)h(x)]p*-1  =  £  h(x)(x  -  l)p*-2.  (11) 

z  =  2  z  =  2 

Writing  x  as  1  +jf,  (r  +  j  where  r  =  1),  eq.(10)  becomes 

oo  oo 

£  M1  +  j)  +  jM*  +  j')]  p"  =  E  Mi  +  jW"1  • 

From  equating  coefficents, 

1.    h{2)  =  0 

2-    g{l  +  j)  +  jh{l  +  j)  =  h{l  +  j  +  l){j  +  1), 

or 

Ml+J  +  1)  =  ?(i±i)±iMi±i) 

J  +  1 
Uj  =  1: 

M3)  =  *M±2  ,  I 

^y  2  2 

If  i  =  2: 

h{4)  =  3 = 3 =  6  =  2  +  3' 


By  induction,  suppose 


Then 


h{l+j+l)     = 


3  +  1 


,       1  1/1  1 


■s-tf  +  i) 


i     i  i 


Thus, 

•x  1  1  1 

Ml+J)  =  5  +  J---  +  -.         J  =  2,3,.... 

Since  /i(X(2))  is  the  unbiased  estimator  for  A  when  r  =  2,  it  follows  that 

0  X(2)  =  2 


M*(2)) 


2^  +  -"xf2T^T   x(2)  =  3'4' 


Case  2:  r  >  1  (by  induction) 
Suppose 

X  is  P(p,r),y  is  P{p,r+  l),g{X)  and  /i(Y) 

are  unbiased  for  —  ln(l  —  p).  Then 

£»w  ( *:  J )  p'-r(i  -p)r  =  £:  *w  ( "; r 

Dividing  out  (1  —  p)r,  expanding  the  resulting  R.S.H.  of  the  equation  and  sup- 
posing ^(r)  =  0,  and  g(r  +  j)  =  J  +  j^  +  . . .  +  ^yn,  we  obtain, 

x-  1  ' 


(1-P)' 


E 

x-r+l 


l_\  )«(*)+     V     M*) 


E 

z=r+l 


/i(x)pz-(r+1). 
(12) 


Setting  x  =  r  +j,  equation  (12)  becomes 


E 


r+£7X     »(r  +  /)+  (r  +  ';-1     Mr  +  i) 


p*  =  E 


r + y  - 1 


/i(r+»p 


J-l 


10 


Equating  coefficients,  we  get 

1.  h(r+  1)  =  0 
2. 


(,;V)«(r+d+(,+j-1U<M-i) 

Mr  +  y+l)=-^  -£-      __     -i—  -X.  (13) 

r:0 


If  J  =  1: 

r         ]<?(r  +  l)+0 
r-1    /    v  r        1  l 


/i(r  +  l  +  l)  = 


r  +  1  \  r(r+  l)r        r  +  1 


If  i=  2: 


(^jU  +  ^+f'^W  +  l  +  l) 

Mr +  1  +  2)     =     ^ '- 


r+:        r+2 
By  induction,  suppose 


1      +-k  ("I 


»('  +  1+»  =  7TT  +  7T2  +  -  +  rb-  (15) 


Then 


fc(r  +  l+j  +  l)  =  -^-       _i_ V_-  (16) 

/    r  +  j  +  1  \ 
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Since, 


r  +  J 
r-  1 


r  +  j+l 

r 


r+j+l 


r  +  J 

r 


r  +  j+l 

r 


equation  (16)  becomes 


i  +  i 

r  +  j+l 


h{r  +  l+j+l)     = 


rg(r  +  j  +  1)  +  (j  +  l)h(r  +  j  +  1) 
r  +  J  +  1 


[r  ^  r+1 


+  ... 


1 

r+J 


+  0'+i) 


r+1  ^   r+2   ^  •  •  •  -T  r+;- 


r  +  J  +  1 


Thus, 


11  11 

+  — —  +  .-•+ :  + 


r+1        r  +  2 


h{r  +  j  +  l) 


r+j       r+j+l 
3  =  0 


1  1 

— :  +  ■■■  +  — r o   J^1*2- 

r+1  r+j 


Since  /i(X(r  +  1))  is  the  unbiased  estimator  for  Ar+i  it  follows  that 

0  X(r+l)  =  r  +  l 

h(X(r +  !))=[        j 


rTI  +  --+X(r  +  l)-l'    ^+D  =  r+2,r  + 


3,.. 


The  induction  has  been  established  for  any  r.   Thus,  if  X(r)  is  P(p,r),  r  >  1, 
then  an  unbiased  estimator,  g(X(r)),  for  —  ln(l  —  p)  is 

g(X(r))  =  i  +  -^  +  . . .  +  ^-—  =  Ar. 


r       r+1 


X(r)  -  1 
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The  corresponding  estimator  for  p  is  pr  =  1  —  exp(-Ar).  Since  ^4r  is  a 
function  of  X  only,  and  since  X  is  a  complete  sufficient  statistic  for  p,  then  from 
the  Lehman-Scheffe  theorem,  Ar  is  a  uniformily  minimum  variance  unbiased 
estimator  for  A  =  —  ln(l  -  p).  Also  E(pr)  <  p  by  using  a  proof  similar  to  the 
one  used  to  show  that  E[p)  <  p. 

Table  3  provides  a  comparison  of  the  statistical  proporties  of  pz  with  those  of 
p  described  at  the  end  of  Theorem  1.  The  comparisons  are  the  result  of  10,000 
computer  replications  of  sampling  from  P(p,r  —  3),  i.e.  testing  until  3  failures. 


p 

E{P) 

E(ps) 

MSE(p) 

MSE(p3) 

.5 

.450 

.452 

.0537 

.0538 

.6 

.548 

.551 

.0516 

.0493 

.8 

.757 

.763 

.0293 

.0248 

.9 

.872 

.876 

.0120 

.0094 

.95 

.933 

.936 

.0043 

.0032 

.99 

.985 

.986 

.0025 

.0017 

Table  3:  Statistical  Characteristics  of  p  and  ps(r  =  3) 

It  is  interesting  to  compare  the  values  of  p  in  Table  2  (r  =  3)  with  those  of 
p  in  Table  1.   (r  —  l). 

3     RELIABILITY  GROWTH  IN  ACQUISITION  PRO 
GRAMS 


Reliability  growth  methods  were  developed  as  early  as  1962,  primarily  for  use 
in  major  DOD  acquistion  programs.  Reference  (1),  presents  several  RG  models, 
including  the  discrete  exponential  (DE)  model.  Reference  (2)  evaluated  many 
reliability  growth  models  in  the  open  literature,  and  other  sources,  at  the  time 
of  the  publication.  It  showed  the  DE  model  had  some  of  the  most  accurate 
characteristics  among  all  growth  models  considered.  Reference  (3)  presents  the 
results  of  numerous  simulations  of  a  modification  to  the  DE  model.  Reference 
(4)  is  a  refinement  and  extension  of  Reference  (3). 
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Reliability  growth  methods  were  used  for  many  years  in  early  development 
phases  of  acquisition  programs  as  a  management  tool  to  indicate  trends  in  growth 
patterns.  They  were  not  used  to  demonstrate  reliability  requirements,  because 
numerous  demonstration  tests  were  performed  for  that  reason.  MILHDBK  189 
(reference  7)  was  developed  under  that  concept.  In  October  1986,  DOD  issued 
reference  (5).  This  document,  recent  RFP's,  and  funding  policies  have  dimin- 
ished the  amount  of  demonstration  testing.  "Reliability  testing"  is  performed 
during  the  development  phases  to  induce  failures,  make  corrections,  and  test 
again.  This  type  of  testing  program  results  in  changes  in  the  design  and  man- 
ufacturing processes.  One  consequence  of  such  a  program  is  the  need  to  track 
growth  in  reliability.  These  new  changes  in  testing  philosphy  and  reliability 
measurement  are  evident  in  references  (5)  and(8). 

4     EXPONENTIAL  RELIABILITY  GROWTH  MODEL 

4.1      Description  of  the  Model 

The  model  assumes  that  a  type  of  component  will  see  several  phases  of  improve- 
ment during  that  portion  of  its  life  cycle  to  which  the  model  is  applied.  Rk 
denotes  reliability  in  the  Kth  phase  or  modification  to  the  type  of  component. 
Specifically,  the  model  is 

RK  =  l-exp{-{a  +  /3K)}  (17) 

for  K  —  0,1,2,....  The  K  —  0  phase  denotes  the  phase  prior  to  any  modifi- 
cation. The  model  assumes  testing  is  performed  in  each  phase  until  a  specified 
number  of  failures  are  observed  at  which  time  a  change  may  be  made.  No  as- 
sumption is  made  about  the  distribution  of  the  time  to  failure.  Only  attributes 
data  are  used  with  this  model.  Estimates  a  and  (3  are  updated  as  new  test  data 
(new  phase)  becomes  available  due  to  testing  subsequent  to  a  modification  in 
the  design  or  manufacturing  process.  This  data  may,  or  may  not,  be  mixed  with 
test  data  from  previous  phases.    All  data  from  all  previous  phases  are  used  in 
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the  estimation  methods  presented  here. 

The  estimates  ock  and  /?#-  for  a  a  and  /?  at  the  end  of  the  Kth  phase  are 
obtained  using  linear  regression  methods  and  an  unbiased  estimator  for  (a+f3K). 
The  data  collected  during  testing  in  the  Kih  phase  is  the  following:  Fk  =  the 
total  number  of  failures  during  the  Kth  phase,  and  NjtK  —  the  number  of  test 
between  the  (j  —  l)st  failure  and  the  jth  failure,  including  the  jth  failure,  in  the 
Kth  phase,  j  =  1,2,..., FK. 

An  unbiased  estimator  Yjk  of  the  quantity  (a  +  (3K)  using  the  jth  sequence 
of  tests  in  phase  K  is  given  by  (Reference  1). 

_  (0  if  NiiK  =  1 

YiK  =  {a  +  0K)jK=      1  +  I+  1       ifJV      >2  (18) 

I  2  NjtK 

for  K  =  0, 1,2, . . .  and  j  =  1,2,3, ... ,  Fk-  Since  each  Yjk  is  unbiased, 

YK  =  (Yik  +  Y2K  +  .-.  +  YFkk)/Fk 

is  unbiased.   Then  least  squares  estimates  ock  and  Pk  for  o:  and  (3  at  the  Kl 
phase  are 

flU-VYj 
Pk  =  ^ (19) 

£0  -  *)2 

;=0 

and 

«K  =  F  -  fo  7f  (20) 

for  ff  =  1,2,3,...  ,  where  7=  (^  +  ^1  +  •  •  •  +  YK)/{K+  1)  and  K  =  (1  +  2  + 
...  +  *)/(#+ 1). 

Finally,  these  ock  and  /?#-  estimates  are  utilized  in  the  discrete  reliability 
model,  equation  (15),  to  produce  sequentially,  the  model  estimates  of  the  modi- 
fied component  reliabilities  i?i,  i?2>  •  •  •  from  the  equation 

RK  =  l-exp{-{&K  +  0KK)}  (21) 
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for  K  —  1,2,3, .. ..  Note  that  since  the  regression  procedure  requires  a  minimum 
of  two  observations,  model  reliability  estimates  are  produced  from  the  first  mod- 
ification thru  the  last  modification.  A  reliability  estimate  for  original  version  of 
the  component  is 

£0  =  exp{-y0}-  (22) 

4.2     FAILURE  DISCOUNTING 

Reduced  testing  support,  new  funding  policies  and  new  DOD  documents,  such  as 
reference  (5),  in  the  acquisition  process  require  maximum  use  of  small  amounts 
of  test  data  to  obtain  reliability  estimates.  Human  judgment  by  both  contrac- 
tor and  customer  personnel  have  played  a  larger  role  in  reliability  assessment  - 
particularly  in  deciding  what  data  to  use  to  estimate  reliability  and  track  reli- 
ability growth.  Failure  analysis  and  review  frequently  generates  a  change  that 
both  contractor  and  customer  feel  have  removed  the  cause  of  failure.  It  may  be 
desireable  to  reduce  the  affect  of  the  original  failure  on  the  reliability  estimate 
since  it  is  believed  to  have  been  "removed" . 

This  raises  the  question  of  what  to  do  with  the  recorded  failure  in  the  data. 
One  method  for  failure  removal  is  to  remove  fractions  of  a  failure,  repeatly,  as 
more  testing  is  accumulated  without  failure  for  the  same  cause.  By  some  rule, 
or  agreement,  fractions  of  such  a  failure  are  removed  until  all,  or  nearly  all,  of 
the  failure  is  removed.  If  the  same  failure  cause  reappears,  the  orginal  failure  is 
restored  and,  perhaps,  the  rule  for  removal  of  fractional  failure  for  this  or  any 
other  cause  may  be  tightened  due  to  the  reduced  confidence  in  ability  to  remove 
failure  causes. 

Accuracy  of  reliability  growth  models  with  or  without  failure  discounting  can 
be  readily  examined  by  computer  simulations.  Several  reliability  growth  models 
have  been  simulated  under  a  variety  of  testing  and  discounting  plans  by  Drake 
(9)  and  Chandler  (10). 
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4.3      Discounting  Methodology 

Once  a  system  weakness  has  been  corrected,  and  improvement  validated  through 
further  testing,  fractionally  discounting  the  past  failure  is  a  method  of  reflecting 
the  improved  system  reliability  in  the  previously  collected  data.  If  a  proper 
discounting  methodology  can  be  developed,  failure  data  from  all  past  phases 
could  be  discounted  and  made  compatible  with  the  data  from  the  current  phase. 
Regardless  which  discounting  method  is  used,  experience  and  good  engineer- 
ing judgment  will  be  necessary  to  choose  input  parameters  to  the  discounting 
method.  If  improper  parameters  are  applied,  the  model  may  consistently  over 
or  underestimate  system  reliability.  Computer  programs  have  been  written  by 
Drake  (9)  and  Chandler  (10)  that  allow  a  user  to  examine  the  affect  of  discount- 
ing on  specific  reliability  growth  models  and  testing  programs.  The  following 
sections  describe  the  specific  steps  for  applying  two  failure  discounting  methods, 
including  examples  of  their  use. 

a.   Straight  Percent  Discounting  Method 

The  Straight  Percent  Discounting  Method  removes  a  fixed  fraction  of  a  failure 
each  time  a  predetermined  number  of  trials  are  observed  without  re-occurrence 
of  the  failure  cause.  The  fraction  of  a  failure  removed,  F,  is  referred  to  as  the 
discount  fraction.  After  the  initial  occurrence  of  a  particular  failure  cause,  any 
trial  in  which  the  failure  cause  is  not  repeated  is  considered  a  success.  The  num- 
ber of  successful  trials  between  applications  of  the  discount  fraction  is  referred 
to  as  the  discount  interval,  N.  Both  N  and  F  are  input  parameters  to  be  speci- 
fied by  the  user  of  the  simulation  programs  written  by  Drake  (9)  and  Chandler 
(10).  They  remain  constant  throughout  failure  discounting  computations.  In 
this  discounting  method,  the  computer  records  the  number  of  successful  trials, 
M,  since  the  last  occurence  of  each  observed  failure  cause.  Failure  discounting 
is  performed  at  the  end  of  each  testing  phase  using  the  values  for  N,  F,  and  M. 
Detailed  explanations  of  this  method  are  provided  in  references  (9)  and  (10). 
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b.  Upper  Confidence  Bound  (UCB) 

The  underlying  premise  of  the  UCB  Discounting  Method  states  that  the 
fraction  of  a  failure  removed  by  the  discounting  method  should  not  be  arbitrarily 
chosen.  Some  statistical  basis  should  be  used  to  determine  how  much  of  a  past 
failure  remains  after  each  successful  test  subsequent  to  failure  and  corrective 
action.  The  UCB  procedures  takes  the  upper  confidence  limit  for  the  probability 
of  failure  to  be  the  fraction  of  the  failure  that  remains.  This  confidence  limit  is 
recomputed  after  each  successful  test  subsequent  to  the  corrective  action  taken 
in  response  to  the  failure.  This  technique  allows  the  user  to  control  the  amount 
by  which  failures  are  discounted  by  specifying  the  level  of  confidence  bound. 
A  single  input  parameter,  the  confidence  level  7,  is  required  to  perform  failure 
discounting.  This  discounting  method  was  proposed  by  Lloyd  (6). 

The  UCB  discounting  equation  is  given  as 

ADJUSTED  FAILURE;  ie,  UCB    =  1  -  (1  -  7)^    for  M  >  0 

=  1.0  for  M  =  0  ' 

5      SUMMARY  OF  ACCURACY  ANALYSIS 

The  simulations  performed  by  Drake  (9)  allowed  for  10  sequential  testing  phases. 
There  were  5  possible  statistically  independent  failure  causes  in  each  phases.  Ini- 
tial failure  probabilities  are  assigned  to  each  failure  cause.  A  single  discounting 
fraction  is  also  assigned.  A  probability  of  correcting  a  failure  causes  whenever  it 
occurs  is  assigned  to  each  failure  cause;  consequently,  the  actual  growth  pattern 
on  any  one  replication  is  random  depending  on  the  actual  failures  and  fixes.  The 
percent  increase  in  reliability  following  a  failure  fix  is  also  an  input  parameter. 
Simulations  are  performed  with  and  without  failure  discounting.  The  number 
of  successive  success  before  discounting  is  applied  is  also  an  input  variable. 

Chandler  (10)  extended  the  computer  program  written  by  Drake  to  permit 
the  user  to  set  the  actual  reliability  in  each  phase.  This  allows  the  user  to  ex- 
amine the  accuracy  of  the  growth  models  for  specific  growth  patterns;  i.e.,  for 

18 


specific  phase  reliabilities  p1)p2,  •  •  •  ,Pio  which  the  user  specifies.  Chandler  ex- 
tends Drakes's  program  in  other  ways  and  examines  features  of  growth  patterns 
for  which  specific  discounting  methods  and  growth  models  are  reasonably  accu- 
rate. Figures  1  through  19  have  been  selected  from  Drake  (9)  and  Chandler  (10) 
to  provide  a  summary  of  the  accuracy  of  the  DE  and  other  reliability  growth 
models. 

The  following  notation  is  used  in  the  Figures  1  through  19  which  are  graphical 
representations  of  the  accuracy  of  the  growth  models  analyzed: 

F:  fraction  of  remaining  failure  removed  each  time  a  discount  is  made. 

I:   number  of  successful  tests  without  failure  before  the  next  failure  discount  can 
be  made. 

Example:  If  F  =  .2  and  1=5,  then  after  a  failure  occurs  and  correction  made, 
the  one  failure  for  that  cause  is  reduced  to  .8  of  a  failure  if  no  failure  for  the 
same  cause  occurs  in  the  next  5  tests.  It  is  reduced  to  .64  if  5  additional  success 
are  recorded.  Anytime  a  failure  for  the  same  cause  re-occurs  the  failure  value  is 
restored  to  1. 

MLE  Single  Phase:  the  name  of  the  growth  model  that  computes  the  relia- 
bility in  each  phase  by  (number  of  successes  that  phases)  -f-  (number  of 
tests  in  that  phase). 

MLEFD:  the  name  of  the  growth  model  that  computes  the  reliability  in  the 
kth  phase  by 

(total  number  successes  in  first  k  phases)  -5-  (total  number  of  tests  in  first 
k  phases). 

Exponential  Regression:  The  discrete  exponential  regression  model. 
Actual  Reliability:  The  reliability  values  assigned  to  each  phase  by  the  user. 
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The  reliability  values  for  each  phase  for  a  specific  growth  model  (e.g.  expo- 
nential regression)  is  the  average  of  500  reliability  values  obtained  by  replicating 
the  10  phases  of  testing  500  times  via  computer  simulations.  Consequently,  the 
points  on  the  growth  "curves"  represent  the  average  or  expected  value  of  the 
reliability  estimates  for  the  actual  reliabilities.  Ideally  the  growth  curves  would 
be  the  same  as  the  actual  reliability  curve  and  the  standard  deviation  would  be 
very  small.  The  standard  deviation  curves  are  the  sample  standard  deviations 
taken  from  the  500  replications. 

The  test  plan  in  Figures  1  through  19  is  to  test  until  1  failure  occurs  in  each 
phase.  This  implies  that  a  change  (correction)  is  made  after  each  failure.  The 
number  of  failure-causes  is  the  same  from  phases  to  phase.  The  change  may 
not  have  totally  removed  the  cause  of  the  failure.  It  may  have  only  reduced  the 
likelihood  that  it  will  occur.  Alternatively,  it  may  mean  that  a  new  mode  of 
failure,  less  likely  to  occur,  was  introduced  with  the  corrective  action. 

Instead  of  5  failure  causes  per  phase,  one  may  wish  to  think  of  a  system  with 
5  components  in  series.  The  program  will  permit  any  number  of  phases  and  any 
number  of  causes  per  phase.  If  it  is  desired  to  simulate  a  case  where  each  failure 
cause  is  removed  with  probability  one,  this  can  be  done  using  Drake's  program 
with  probability  of  fix  equal  to  one  and  the  number  of  phases  less  than  or  equal 
to  the  number  of  causes. 

Figures  1-19  present  graphical  results  of  simulations  for  6  different  growth 
patterns  each  with  various  combinations  of  I  and  F,  and  two  values  of  confidence 
level  for  the  Lloyd  discounting  method.  Recall  that  1=1  in  the  Lloyd  discounting 
method. 

In  general  the  following  conclusions  can  be  from  Figures  1-19. 

1)  The  MLEFD  Method  is  highly  sensitive  to  failure  discounting.  This  method 
is  commonly  used  in  industry. 

2)  The  exponential  regression  method  is  consistently  more  accurate  than  the 
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other  models  and  is  not  very  vulnerable  to  failure  discounting. 

3)   Failure  discounting  improves  the  exponential  regression  method  in  many 
cases. 
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Figure  1. 
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Figure  2 
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Figure  3. 
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Figure  4. 
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Figure  5. 


26 


<c 

o 

fc  o 
_j 

< 

_J 

LJ    «* 

*■  o 

—  a  — 

ACTUAL  RELIABILITY 

-0- 

MLE  SINGLE  PHASE 

-<?- 

EXP  REG 

d 

-  +  - 

MLEFD 

PHASE 


10 


2                        4                         6 

PHASE 

8 

10 

2                         4                        6 

PHASE 

.     Pittern  II,  F  -  .50,  I  -  3 

Pattern  II,  F  -  .50,  I  -  6 

Figure  6. 


27 


PHASE 


£3- 


/    •'■*- 

I                    !                    1 

-•-  ACTUAL  REUABIUTY 
-D-  MLE  SINGLE  PHASE 

-V-  EXP  REG 
-  +  -  MLEFD 

1                  1                  !                  1                  1 

1 

PHASE 


PHASE 


10 


Pattern  II,  F  -  .75,  I  -  3 


Pattern  II,  F  -  .75, 1  -  6 


Fifure  7. 


-•-  ACTUAL  RELIABIJTY 
-C-   WLE  SINGLE  PHASE 

-V-  EXP  REC 
-  +  -  WLEFD 


PHASE 


PHASE 


1C 


PHASE 


10 


-.      Pattern  II,  Lloyd,  CI  -  .8 


Pattern  II,  Lloyd,  CI  -  .9 


Figure  8. 


29 


ACTUAL  RELIABILITY 
WLE  SINGLE  PHASE 
EXP  REG 
-+-  ULEFD 


10 


PHASE 


PHASE 


Pattern  IV,  No  Discounting 


Pattern  IV,  F  -  .25, . 


25 


Figured. 


30 


-•-  ACTUAL  RELIABILITY 
-D-   MLE  SINGLE  PHASE 

-V-  EXP  REG 

-  +  -  MLEFD 


J L 


PHASE 


PHASE 


1C 


10 


I 


-•-  ACTUAL  RELIABILITY 
-D-  MLE  SIMGLE  PHASE 
-V-  EXP  REG 
-  +  -  MLEFD 


PHASE 


IC 


•: 


Pitteni  IV,  Lloyd,  CI  -  .3 


Pittern  IV,  Uoyd,  CI  -  .9 


Fifure  10. 


SI 


•  -  ACTUAL  REUA3iLITY 
D-  ULE  SiNGLE  PHASE 

V-  EXP  REC 
+  -  WLEFD 


■♦-  ACTUAL  REUA3IUTY 
D-  MLE  SINGLE  PHASE 

V-  EXP  REG 

+  -  MLEFD 


PHASE 


PHASE 


Pattern  V,  No  Discounting 


Pattern  V,  F  -  .25,  I  -   1 


Figure  11. 


32 


-»-  ACTUAL  RELIABILrTY 
-D-   ULE  SINGLE  PHASE 

-V-  EXP  REG 

-+-  MLEFD 


PHASE 


PHASE 


10 


10 


Pattern  V,  Lloyd,  CI  =  .S 


Pattern  V,  Lloyd,  CI  -  .9 


Figure  12. 


S3 


b  o 
m 


, — i — 
— ? — 

— ^~~ 

— * — 

* 

- 

— e — 

— € — 

6 — 

g_ o e e — 

— B 

Q 

- 

- 

-•-  ACTUAL  RELIABILITY 
-D-  MLE  SINGLE  PHASE 

-V-  EXP  REG 

-  +  -  WLEFD 

- 

t 

i             •             t             i 

! 

Q 
< 


PHASE 


PHASE 


10 


10 


CD 

< 

UJ   •* 
K  o 


e « 


-•-  ACTUAL  RELIABILITY 
-D-  WLE  SINGLE  PHASE 
-V-  EXP  REG 
-+-  MLEFD 


10 


PHASE 


PHASE 


Pattern  VI,  No  Discounting 


Pattern  VI,  F  -  .25,  I  -  1 


Figure  13. 
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Figure  14. 
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Figure  16. 
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